Papers with token classification tasks
Selective Prefix Tuning for Pre-trained Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained models are time-consuming and memory-inefficient. |
| Approach: | They propose a method that inserts learnable vectors into each Transformer layer . they propose SL to encourage diversity in prefix tokens . |
| Outcome: | Extensive experiments validate the effectiveness of Prefix Tuning in sentence and token classification tasks. |
ChuLo: Chunk-Level Key Information Representation for Long Document Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Traditional approaches to truncate inputs, sparse self-attention, and chunking often lead to information loss and hinder the model’s ability to capture long-range dependencies. |
| Approach: | They propose a novel chunk representation method that uses unsupervised keyphrase extraction to group input tokens to retain core document content while reducing input length. |
| Outcome: | The proposed method minimizes information loss and improves the efficiency of Transformer-based models. |
People and Places of Historical Europe: Bootstrapping Annotation Pipeline and a New Corpus of Named Entities in Late Medieval Texts (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained named entity recognition models are inaccurate on modern corpora due to differences in language OCR errors. |
| Approach: | They develop a named entity recognition (NER) corpus of 3.6M sentences from medieval charters written mainly in Czech, Latin, and German. |
| Outcome: | The proposed model achieves entity-level Precision of 72.81–93.98% with 58.14–81.77% Recall on a manually-annotated test dataset. |
The Devil Is in the Word Alignment Details: On Translation-Based Cross-Lingual Transfer for Token Classification Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Translation-based strategies for cross-lingual transfer XLT include label projection . word aligners (WAs) are commonly used for label projection, but low-level design decisions for using them have not been investigated . |
| Approach: | They revisit word aligners (WAs) for label projection and propose a new projection strategy that outperforms WAs. |
| Outcome: | The proposed projection strategy outperforms marker-based methods in token classification tasks. |
TransAlign: Machine Translation Encoders are Strong Word Aligners, Too (2025.findings-emnlp)
Copied to clipboard
| Challenge: | translation-based approaches to cross-lingual transfer (XLT) are limited. |
| Approach: | They propose a word aligner that utilizes the encoder of a massively multilingual MT model. |
| Outcome: | The proposed word aligner outperforms existing WA and state-of-the-art non-WA-based methods in token classification tasks. |